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In recent times, malware has progressed by utilizing distinct advanced 
machine learning techniques for detection. However, the model becomes 
complicated and the singular value decomposition and depth-based malware 
detectors failed to detect the malware significantly with minimum time and 
overhead. This paper proposes a multinomial linked latent dirichlet and 


modular double q learning (MLLD-MDQL) to efficiently detect malware 
based on the network behavior patterns. First, multinomial linked latent 
Keywords: dirichlet network behavior extraction (ML-LDNBE) is applied to the input 
network for anomaly detection that extracts the behavior based on the 
network pattern. The behavior is extracted which are grouped to perform on 
the path protocol for analyzing repeated behaviors. Finally, the modular 
double q learning malware classification model is the grouped behaviors for 
Multinomial significant malware detection. The effectiveness of proposed MLLD-M 
Network behavior DQL method is compared with existing models. The results obtained by the 
proposed method show that the model combined with the machine learning 
(ML) significantly determined malware detection time and also reduced the 
false positive rate (FPR). The results showed that the false positive rate is 
significantly reduced by 42% for the proposed method better when 
compared to the existing behavior based malware detection model that 
obtained 62% of FPR. 
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1. INTRODUCTION 

In computer security, malware detection is prospering due to its devastating intents. In recent years, 
it has become an immense ultimatum to make the system secure from malware attacks [1]. The rapid growth 
of the malware complication was having an issue for both computer security and the network [2]. In the 
existing research Khraisat et al. [3], three prominent features were processed to develop a malware classifier 
model [4]. Initially, strings in printable formats were processed through mining results the high dimensional 
features which are on the string. Next, to reduce the dimension involved, singular value decomposition was 
applied [5]-[7]. Shannon entropy was evaluated to consider both the randomness of the application 
programming interface (API) and printable string information (PSI) features [8], [9]. Moreover, behavioral 
features such as key modification, file operations regarding the registry and network activities for malware 
detection [9]-[11]. The features are combined and summed up for training the feature set emerged with the 
machine learning-based algorithms for malware classification [11]-[13]. With this, the malware detection 
accuracy was recorded to be high [14]. 
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A malware based on behavior chains was proposed in [15]. The proposed method monitored the 
behavior points based on calls made by API. Then the model utilized the calling progression of the behavior 
points performed at runtime to assemble a behavior chain [16]-[18]. Finally, a depth detection model has 
utilized based on long short term memory that in turn differentiated between malicious behaviors that were 
observed from the behavior chains [19]-[22]. With this, the malware detection accuracy was said to be 
improved with a minimum false positive rate [22]-[24]. 
This paper introduces a method to detect malware attacks. In this paper, a behavior-based machine 
learning technique called multinomial linked latent dirichlet and modular double q learning (MLLD-MDQL) 
is designed [25]-[29]. The model takes advantage of network behavior pattern extraction and behavior 
classification. The existing researchers used ML models for malware classificiaiton and analyzed the 
problems arose in them. Singh and Singh [30] detected the malware software to analyze based on the artifacts 
using ML algorithms. The malware detection was performed based on the behavior of the research. The run 
time features were extracted by analyzing the dynamic environment based on the Cuckoo search algorithm. 
However, the developed model failed to achieve better efficiency concerning the context of the processed 
features. 
Similarly, Zhang et al. [31] developed a depth detection model for malware detection based on the 
behavior chains. The developed model was related to the behavioral points and has proposed a depth 
detection method with respect to the behavior chains. The developed model monitored the behavior based on 
the API calls and designed those points in the calling sequence at the run time for behavior chain 
construction. The developed model detected the malicious behavior based on the long short term memory 
(LSTM) that required to analyse the model based on the behavior chain. Additionally, Jeon et al. [32] 
developed a convolution neural network (CNN) for dynamic analysis and detection of IoT malware. The 
developed DAIMD scheme learned the IoT malware using the CNN-based model for analyzing the cloud 
environment. Thus, the implementation of the model was important to detect the IoT malware based on the 
hybrid analysis technique that utilized both dynamic and static techniques further. 
Similarly, Zhao et al. [33] developed a hybrid Gram model for feature extraction to detect 
malicious behavior using machine learning (ML). The developed model was constructed based on the 
dynamic feature for malware analysis as it used a novel feature extraction method for the H-gram to evaluate 
the cross-entropy. It provides the continuous overlapping of subsequences that implements the semantic 
segmentation for the API calls and instructions. However, the extracted feature sequence was not enough that 
show a lower degree of discrimination. Also, Liu et al. [34] developed a machine learning algorithm for 
automated malware classification and the decision making utilizes the features for the classification. The 
developed model identified the suspicious malware and classified the new malware. The detection modules 
used shared nearest neighbor (SNN) based model for clustering and to discover the families of the new 
malware. 
The results showed that the developed model improved the accuracy for the effective classification 
but failed to discover the new malware effectivel and thus, Pattee et al. [35] developed an intelligent malware 
detection and design alternatives for performance monitoring counter. The developed model improved the 
importance of selecting the features for malware detection that showed statistical differences among the 
malware workloads and benign workloads were characterized on the basis of information from counters 
performances. However, the detailed architectural design for the dedicated accelerator was required for 
providing efficiencies with the chip area power, and processing time. Also, Botacin et al. [36] developed a 
signature based malware detection system for accelerating the hardware-enhanced antivirus engine 
(HEAVEN). The workflow consisted of a hardware-assisted signature that matched the process of first step 
(triage) that is fast, and invokes with the software-based AV. At the time of software suspiciousness an 
unknown hardware signature is generated for malignity. The mechanisms were claimed with flexibility than 
HEAVEN as it was not limited to signature matching. Yet, the model was not often discussed when it comes 
for flexibility showed improvement in terms of performance cost that was not required as the signatures were 
not enough. 
The multinomial linked latent dirichlet (MLLD) to extract network behavior and then classify by 
applying modular double q learning (MDQL). In short, the main contributions are as follows: 
To construct a novel network behavior pattern model known as MLLD network behavior extraction 
model that combined linked latent dirichlet allocation with behavior graphs of API for malware 
detection. 

- The MLLD-MDQL was developed to analyse the modular double q learning classification model which 
automatically acquires high-level representations of malware behaviors. 

- To obtain experimental results exhibiting the proposed method extracts more relevant conceptual 
features and helps to reduce the time, overhead and false-positive involved in malware detection. 
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The organization of the research paper is given as follows: Section 2 explained the proposed method 
with malware behavior. Section 3 shows the results for the proposed method which analyzes the malware 
behavior for the simulation of the results. The conclusion of this research is given in Section 4. 


2. RESEARCH METHOD 

The behavior-based malware detector on the other hand distinguishes network behaviors with the 
aid of monitoring tools and decides whether the program is malware. The MLLD-MDQL is split into two 
stages which are observed and obtain the dimensionality reduced network pattern behaviors. Next, malware 
attack detection is made by means of a robust classification model. A detailed description of the proposed 
method is followed with the below dataset description. 


2.1. Dataset description 

In this work, the malware detection based on the network behavior patterns is made by applying 
https://www.kaggle.com/anushonkar/network anamoly-detection dataset. The dataset includes basic features, 
‘BF = {bf,, bfz, ..., bfo}’, content features ‘CF = {cf,,cfy,....,cfy3} and time-related traffic features 
‘TTF = {ttf,, ttfy,.., ttfio}’ respectively. 


2.2. Multinomial linked latent dirichlet network behavior extraction 

Network behavior cites the pursuits of both the network and the users’ control to work on it. To 
accurately measure its network’s security, network behavior patterns have to be analyzed and observe for any 
malware that stipulates a security threat. This malware detection through network behavior patterns not only 
assists in alleviating security issues but also inspects present-day and prevailing behavior to obtain a 
comprehensive sketch of network security. 

In this section, a malware detection using behavior association is presented. The network behavior 
patterns are extracted with the aid of the MLLD model. Figure 1 shows the block diagram of the multinomial 
linked latent dirichlet network behavior extraction (ML-LDNBE) model. 

As illustrated in Figure 1, the input is acquired from https://www.kaggle.com/anushonkar/network- 
anamoly-detection dataset. The main objective of the above model remains constant in extracting the 
behavior based on the network patterns of malicious code based on the API monitoring model fuses the 
behavior. Next, with data dependence, an association between behaviors is confirmed. 

Finally, the MLLD allocation model is utilized in extracting semantically and globally related 
behaviors from sample programs that permit the set of observations. This is obtained by hidden groups that 
explain why certain behaviors of the network traffic data are similar. The linked type of allocation is 
performed by using multinomial latent dirichlet for behavioral network-based feature extraction. The faster 
access time with less memory overhead better the ensure. 


Data dependence 


Behavior association 
established 


Linked Latent 
Dirichlet allocation 


Extract 
semantically and 
global related 
behaviors 


Figure 1. Block diagram of multinomial linked latent dirichlet network behavior model 
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Let us consider a digraph ‘G(V, E)’ to denote the behavior association. Here, the node ‘V;’ denote 
the behavior and a directed edge (V; V;)’ denotes the data dependence relationship, where ‘V; > V;’ denotes 
‘Vp? that depends on ‘V;’. Network behavior refers to multiple approximately close behavior patterns, like, the 
number of file creation, source bytes, duration and so on. To ease the interpretation of the upcoming x 
algorithm, let us utilize a 4-tuple behavioral pattern model, which is mathematically expressed in (1). 


A = {BS,IB,OB,CB} (1) 


From (1), ‘BS’ denotes the behavior states (i.e., success, failure, or idle state), ‘JB’ denotes the input 
behavior, ‘OB’ denotes the output behavior and ‘CB’ represents the combination behavior respectively. The 
behavior state is an API where the program is called to accomplish a task as shown in Figure 1, the ML- 
LDNBE in the present work is used to extract the dimensionality reduced network behavior features (i.e. 
patterns). The ML-LDNBE model assumes that each time-related feature represents the probabilistic 
distribution through the latent distribution. The content is performed all the time which is related to features 
shared before common Dirichlet. 

Each of the latent content is related to the ML-LDNBE features which are represented with the 
probabilistic distribution over a basic level of features which are having basic distributions for sharing the 
content in common to a prior. Thus, the content of the network vectors was linked statistically for linking the 
structure among basic and content features. The time-related traffic and content features are therefore 
summarizing the dimensionality-reduced network behaviors. 

Given a set of network connection vector ‘V € BF,CF,TTF’ consisting of ‘n’ traffic features, with 
network connection vector ‘n’ having ‘Va’ traffic features, ML-LDNBE models ‘V’ according to the 
subsequent generative procedure. Let us select a multinomial distribution ‘0,’ for content features ‘cf (cf E 
{1,2, ...,CF})’ from a Dirichlet Distribution with criterion ‘æ’, multinomial distribution ‘@,;.¢’ for time 
features ‘ttf (ttf € {1,2,..,7TF})’ from a Dirichlet Distribution with criterion ‘£’ and finally, a 
multinomial distribution ‘@,¢’ for basic features ‘bf (bf € {1,2, ..., BF})’. Moreover, content feature ‘6,’ 
from ‘@,,’ and a basic feature ‘6,’ from ‘Øp’. 

In the above generative process, basic features in a network connection vectors are observed with 
the other variables that are latently represented as ‘Ø’ and hyperparameters ‘a’ and ‘p’ respectively. To 
obtain the dimensionality reduced network behavior feature pattern, an exchangeability hypothesis is utilized. 
The hypothesis assumes that the integrated distribution between time and content related features, content 
and basic network features is uniform to permutation. Here, to handle the dimensionality reduced network 
behavior feature pattern, the API function is used and the behavior is said to be handled. 

Let us assume that ‘P’ is a permutation of the network features from ‘1’ to ‘N’, then, a finite set of 
random network features ‘{cf,,cfo,...Cfm}’ are also said to be interchangeable. This is mathematically 
expressed as given in (2). 


P (ch, Chor s Chm) = P(Chncay Cfa = fem) ) (2) 


From (2), ‘a(.)’ refers to the permutation function on the network features ‘{1,2,...,m}’. Then, the 
probability of observed network data ‘OD’ is evaluated as given in (3). 


Prob(6;|rrr) 
-m™ Hd ; 
Prob(OD|a, B) = []%4; Prob (Orrela) | 1, Prob(6,|5;, Bid; 


From (3), the generative process correlates to the cooperative distribution of the latent content 
| 141 Prob(6;|Orrr) Prob(d;|6;, B )id;]’ and observed variables ‘[]/4,Prob(@rrr|a)’ for time related 
features. The pseudo code representation of Multinomial Linked Latent Dirichlet Network Behavior is given 
as Algorithm 1. 

As given in the above Multinomial Linked Latent Dirichlet Network Behavior extraction for 
malware detection, network behavior patterns are analyzed using content related, basic and time-related 
traffic features. With these features as input, first, a 4-tuple behavior pattern is formulated. Followed by 
which, permutation of network features is performed for a finite set of random network features. Finally, with 
linked data dependence, dimensionality reduced network behavior features (i.e. patterns) are extracted with 
minimum time and overhead. 
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Algorithm 1 multinomial linked latent dirichlet network behavior 

Input: basic features ‘BF = {bf,,bfy,...,bfo}’, content features ‘CF = {cfi,cfy,...,cfi3}’, time-related 
traffic features ‘TTF = {ttf,,tthy,..,ttfio}’ 

Output: Network behavior features ‘B’ 

Step 1: Initialize content feature multinomial distribution ‘Oy’, time feature multinomial 
distribution ‘Og’, basic feature multinomial distribution `‘Øpf', criterion ‘B’, ‘a’ 

Step 2: Begin 

Step 3: For each basic feature ‘BF’, content features ‘CF’ and time-related traffic 
features ‘TTF’ 

Step 4: Formulate 4-tuple behavioral pattern model using Eq. (1) 


Step 5: Evaluate permutation of the network features using Eq. (2) 
Step 6: Obtain probability of observed network data ‘OD’ using Eq. (3) 
Step 7: Return dimensionality reduced network behavior features (B) 
Step 8: End for 

Step 9: End 


2.3. Modular double q learning malware classification model 

With the extracted network behavior patterns, malware attack detection is made in this work by 
applying modular double q learning malware classification model. Here, the grouping is performed which is 
followed with the classification approach. Grouping is performed based on path and protocol type for 
learning repeated behaviors. By considering what action to be taken under what circumstances, it does not 
require a process of the environment, without requiring adaptations. With double q learning, the classification 
is performed for different policies, where the value referring to the behavior is evaluated reduces false- 
positive rate. 

As the number of agents (i.e. connection in network connection vector) increases, the exponential 
values also increase in the overall dimensions (i.e. number of connections), increasing the memory 
drastically. To address this issue, the modular function is utilized that splits a larger number of connections 
into sub-problem. In the action selection stage, each learning module provides Q-values for corresponding 
actions. The behaviors ‘B’ and the paths (i.e. service ‘Service’ from basic features), protocol (i.e. protocol 
type ‘Protocoltype’ from basic features) are identified. The score is designated based on the protocol 
(‘Protocoltype’) and path (‘Service’). For instance if ‘a; E V;(i.e.,network connection vector)’ 
consists of ‘B,, B3, ..., Bn’ behaviors. 


By(SP)+B2(SP)+---+Bn (SP) 
n 


a;(SP) = (4) 
By (SS)+Bo(SS)+-+-+By (SS) 


n 


a;(SS) = (5) 

From (4) and (5), ‘SP’, ‘SS’ refers to the score for protocol type and score for service respectively 
with ‘B1, B2, ..., Bn’ representing the behaviors. Scores are calculated for each behavior. The score is 
categorized from ‘1’ to ‘5’, where ‘1’ refers to the associated behavior which is normal, and ‘5’ refers to that 
the associated behavior which is risky and probably is malware. 

With the behavior grouped results, next, Double Q Learning is applied to the measured score for 
respective service ‘a;(SS)’ and protocol ‘a;(SP)’ to return different classification results. Therefore, four 
different types of malware classifications are made efficiently. From the name itself, Double Q Learning two 
estimators (i.e., service ‘a;(SS)’ and protocol ‘a;(SP)’) are utilized instead of one estimator for each state- 
action pair where the two values ‘QP’ and ‘QS’ are used. 

Here, the action ‘a’ is picked and based on ‘QP(s,.)’ and ‘QS(s,.)’, the final reward ‘r’ and state 
‘s’? are obtained. To start with either ‘P?’ or ‘S, the values are updated. If an update is made on ‘P’, then 
corresponding action is represented as shown in (6). 


a’ = ARGMAX QP (s',a) (6) 
By finding the action ‘a’’ from the above Eq. (6), next ‘QP’ is updated and is mathematically formulated as 
given in (7). 

QP(s,a) = QP(s,a) + a[R +y QS(s', a’) — QP(s, a)] (7) 


Similarly, if an update is made on ‘S’, the corresponding action is represented as shown in (8). 


b' = ARGMAX QS (s’,a) (8) 
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By finding the action ‘b” from the above Eq. (8), next ‘QS’ is updated and is mathematically formulated as 
given in (9). 


QS(s,a) = QS(s,a) + a[R+y QP(s',a’) — QS(s,a)] (9) 


With the above actions evolved, four different types of malware are detected. As given in the above 
modular double q learning algorithm, two processes are performed. They are behavior grouping and malware 
classification. First, behavior grouping is done via modular function based on two estimators, protocol and 
service. Next, malware attack detection is made using double q learning separately for protocol and service. 
Condition checking with the respective threshold is validated to observe malware attack detection in the early 
stage. Therefore, by applying different policies for each update, accurate detection is said to be made with a 
minimum false positive rate. 


3. RESULTS AND DISCUSSION 

In this section, experimental analysis of the multinomial linked latent dirichlet and modular double q 
learning (MLLD-MDQL) method for malware attack detection is presented. In this section, the performance 
of the proposed MLLD-MDQL is compared with the state-of-the-art methods, Behavior-based malware 
detection [30] and Malware based on behavior chains [31] using network anomaly detection dataset and 
implemented in Java. Table 1 results attack detection time observed for three different methods, MLLD- 
MDQL, behavior-based malware detection [30] and malware based on behavior chains [31] and Figure 2 
shows attack detection time graph with different connections. 


Table 1. Attack detection time results using MLLD-MDQL, behavior-based malware detection and malware 
based on behavior chains 


Count Attack detection time (ms) 
MLLD-MDQL _ Behavior-based malware detection Malware based on behavior chains 
15 6.825 9.375 11.025 
30 8.15 10.435 13.455 
45 9.355 11.315 16.135 
60 11.215 15.435 19.235 
75 13.345 17.135 21.355 
90 15.255 19.215 24.55 
105 18.135 21.435 28.155 
120 21.435 25.555 30.355 
135 23.455 28.155 33.455 
150 25.525 30.235 35.155 
1000 
900 
8 800 
= 700 
a 
g 600 —=MLLD-MDQL 
Z 500 
= =i Behavior-based 
£ 400 malware detection 
be 300 == Malware based on 
= behavior chains 
= 200 
< 
100 
0 
15 30 45 60 75 90 105 120 135 150 
Count 


Figure 2. Attack detection overhead graphs with different connections 
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3.1. Quantitative analysis 
3.1.1. Case 1: performance measure of attack detection time 

The behavior based malware attack detection with respect to detection time is evaluated at 
significantly. The attack detection is referred to as the time consumed in malware detection based on the 
network behavior aspect. Mathematically, the equation is formulated as shown in (10). 


ADT = Yt, Count; x Time [AD] (10) 


From (10), the malware attack detection time based on the network behavior aspect ‘Time [AD]? is 
measured based on the number of connections established as ‘Count; and are measured in terms of 
milliseconds. Table 1 results attack detection time observed for three different methods, MLLD-MDQL, 
behavior-based malware detection [30] and malware based on behavior chains [31] and Figure 2 shows 
attack detection time graph with different connections. 


3.1.2. Case 2: performance measure of attack detection overhead 

The second metric analysed in this research work is the attack detection overhead. The attack 
detection overhead represents the overhead incurred while detecting malware attacks. The mathematical 
expression for the attack detection is expressed as shown in (11): 


ADO = Yi, Count; x Mem [AD] (11) 


From (11), the attack detection overhead ‘ADO’ is obtained through many connections used to 
simulate purpose ‘Count;’ and the memory consumed for detecting attacks ‘Mem [[AD]]’ during attack 
detection. The term kilo bytes (KB) is used to evaluate the model. The results obtained for the attack 
detection observed overhead for 3 distinct methods as MLLD-MDQL, behavior-based malware detection 
[30] and malware based on behavior chains [31]. 

In Figure 2, performances of the attack detection overhead are shown using line graphs for three 
different methods such as MLLD-MDQL, behavior-based malware detection [30] and malware based 
behavior chains [31]. The reason behind the improvement is the application of the Multinomial Linked Latent 
Dirichlet Network Behavior extraction algorithm. By applying this algorithm, network behavior patterns are 
analysed through content, basic and time-related traffic features. By considering these features, a 4-tuple 
behavior pattern is organized and permutation is performed for a finite set of random network features, 
resulting in linked data dependence. With this, the malware attack detection overhead using MLLD-MDQL is 
said to be reduced by 18% compared to [30] and 35% compared to [31]. 


3.1.3. Case 3: performance measure of false positive rate 

In malware attack detection, when multiple comparisons are performed with respect to the FPR 
which is the probability for falsely rejecting the model to obtain the null hypothesis. The FPR has calculated 
the ratio among the number of negative events. The normal connections are identified wrongly are known as 
malware evaluated among the total number of actual negative events (i.e. total number of connections 
involving malware attack). This is mathematically expressed as given in (12). 


FP 
FP+TN 


FPR = 


(12) 


From (12), the false-positive rate ‘FPR’, is measured based on the number of false positives ‘FP’ 
and a number of true negatives ‘TN’ respectively. It is measured in terms of percentage (%) shows the results 
of the false positive rate observed for three different methods, MLLD-MDQL, behavior-based malware 
detection [30] and malware based on behavior chains [31]. Finally, Figure 3 depicts the attack detection 
overhead for different connections. Hence, by applying different policies for each estimator, false positive 
rate is significantly reduced by 42% for the proposed method better when compared to the existing Behavior 
based malware detection model that obtained 62% of FPR. 


3.2. Comparative analysis 

The Table 2 shows the comparative analysis for the proposed MLLD-MDQL that is compared with 
the existing models. The performances are in terms of FPR as 12.85%, time as 25.52 ms, and overhead of 60- 
600 kB for the proposed method. However, the more detailed architectural design for a dedicated accelerator 
provided efficiencies better for chip area, power, and processing time was required to be investigated. 
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However, the developed model when comes under performances cost, it required still more improvement as 
cases signature were not enough. 
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h 


10 —=te= Malware based on 
behavior chains 
5 
0 


15 30 45 60 75 90 105 120 135 150 
Count 


Figure 3. Attack detection overhead graphs with different connections 


Table 2. Comparative analysis 


Method FPR (%) Time Overhead 
Behavior-based malware detection [30] 3.17 - - 
Malware based on behavior chains [31] - 90s - 
PMC-based malware detection [35] 12.5 6.825ms - 
MALDC [36] - 20s 10% 
Proposed MLLD-MDQL 12.85 25.525ms_ 60-600kB 


4. CONCLUSION 

The present research work discusses the new malware attack detection using the proposed MLLD- 
MDQL model. The main contributions of the proposed MLLD-MDQL method for malware attack detection 
reduce the malware attack detection time. The detection time is evaluated overhead and a false positive rate is 
involved. The proposed method reduces the attack detection time and overhead for operating the malware 
detection through multinomial linked latent dirichlet network behavior Extraction that extracts the network 
behavior patterns by monitoring API through data dependence. Next, with behavior grouping and 
classification is performed using modular double q learning malware classification, a false positive rate 
improves the results significantly. The simulations were conducted and the performance was evaluated to 
analyze malware attacks in terms of attack detection time, overhead and false positive rate. The results 
showed that the false positive rate is significantly reduced by 42% for the proposed method better when 
compared to the existing Behavior based malware detection model that obtained 62% of FPR. However, in 
the future Monitoring system events for malware prevention has to be determined. 
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